Random evolutions with underlying semi-Markov processes

The KTH Visit in Semi-Markov Processes

Semi-Markov approach to continuous time random walk limit processes

Continuous time random walks (CTRWs) are versatile models for anomalous diffusion processes that have found widespread application in the quantitative sciences. Their scaling limits are typically non-Markovian, and the computation of their finitedimensional distributions is an important open problem. This paper develops a general semi-Markov theory for CTRW limit processes in d-dimensional Eucl...

Semi-markov Decision Processes

Considered are infinite horizon semi-Markov decision processes (SMDPs) with finite state and action spaces. Total expected discounted reward and long-run average expected reward optimality criteria are reviewed. Solution methodology for each criterion is given, constraints and variance sensitivity are also discussed.

Quantum semi-Markov processes.

We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes. General conditions for the complete positivity of the corresponding quantum dynamical maps are formulated. The resulting non-Markovian quantum processes allow t...

Semi-Markov Decision Processes

The previous chapter dealt with the discrete-time Markov decision model. In this model, decisions can be made only at fixed epochs t = 0, 1, . . . . However, in many stochastic control problems the times between the decision epochs are not constant but random. A possible tool for analysing such problems is the semiMarkov decision model. In Section 7.1 we discuss the basic elements of this model...